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EXECUTIVE  SUMMARY 


INTRODUCTION 

The  current  Department  of  Defense  (DoD)  emphasis  placed  on  the  advancement  of  space 
technology  and  capabilities  has  produced  an  accelerated  rate  of  change  within  the  space 
operations  community.  Along  with  the  accelerated  enhancement  of  capabilities  come  numerous 
task  performance  challenges  for  the  space  operator  who  must  maintain  proficiency  despite 
continual  change  and  increasing  complexity.  Because  space  operators  are  facing  these 
challenges  and  because  their  proficiency  is  critical,  a  training  system  was  designed  to  help  the 
training  of  satellite  operators  -  operators  working  with  the  Space-Based  Infrared  System 
(SBIRS),  in  particular  -  keep  pace  with  the  operational  environment.  This  report  describes  the 
design  phase  of  an  effort  to  enhance  SBIRS  training  for  satellite  systems  crews  by  (1)  increasing 
the  fidelity  of  SBIRS  simulation  training  environments  through  the  addition  of  critical  space 
operations  center  support  applications  and  (2)  providing  instructors  and  trainees  with  training 
support  tools. 

APPROACH 

A  training  requirements  analysis  was  performed  to  identify  skills  and  knowledge  that  should  be 
targeted  by  the  training  system  to  be  developed.  Skill  and  knowledge  training  requirements 
were  identified  based  on  guidance  from  SBIRS  job  performance  requirements  lists  (JPRLs)  and 
subject  matter  experts  (SMEs).  This  analysis  also  identified  challenging  design  issues  that 
relate  to  developing  a  training  system  that  can  be  used  in  an  integrated  manner  with  existing 
training  systems,  and  to  using  technology  and  training  strategies  optimally  to  support  the 
identified  training  requirements  within  a  simulation-based  training  facility.  A  conceptual  design 
of  a  training  system  was  developed  to  target  the  identified  skill  and  knowledge  areas  and  to 
address  the  identified  design  issues.  This  design  will  be  improved  and  refined  through  iteratively 
storyboarding  it  and  seeking  SME  feedback. 

PHASE  I  CONCEPTUAL  DESIGN 

During  this  Phase  I  research  and  design  effort,  a  conceptual  design  for  a  training  system  called 
the  Training  Environment  Support  System  (TESS)  was  developed.  TESS  is  designed  to  help 
the  space  community  keep  pace  with  rapidly  changing  operations  by  (1)  adding  applications  that 
are  used  to  support  SOC  operations  to  SBIRS  training  facilities  and  (2)  providing  training 
support  tools  for  instructors  and  trainees.  More  specifically,  TESS  will  consist  of  cloned  versions 
of  the  terrestrial  weather,  space  weather,  scheduling,  and  messaging  applications  used  in  the 
SOC.  Further,  the  TESS  design  specifies  a  number  of  training  support  tools.  These  tools 
support  scenario  development,  execution,  and  management,  help  instructors  monitor  trainee 
performance,  and  help  instructors  provide  individualized  attention  to  trainees  during  training 
exercises. 

FUTURE  WORK 

Future  work  will  involve  working  with  space  operations  training  personnel  to  iteratively  refine 
the  functionality  and  user  interface  design  of  TESS,  and  then  to  iteratively  implement  the  training 
system  design  specifications.  A  significant  portion  of  this  effort  will  involve  assessing  the  skills 
and  knowledge  associated  with  each  of  the  support  applications  contained  within  TESS,  and 
developing  knowledge-based  training  content  (e.g.,  scenarios,  agent  feedback,  and  MEL 
content)  that  targets  their  acquisition. 
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TRAINING  ENVIRONMENT  SUPPORT  SYSTEM  (TESS); 
PHASE  I  FINAL  REPORT 


INTRODUCTION 

Transformation  in  Space  Operations 

In  the  current  threat  environment,  in  which  military  action  is  often  constrained  by  cultural 
and  population  factors,  and  in  which  threats  are  often  characterized  by  small  signatures,  short 
durations,  and  unpredictability,  space  has  emerged  as  a  dominant  and  essential  contributor  to 
military  operations  and  global  situational  awareness.  Space  systems  and  capabilities  represent 
key  objectives  and  enabling  technologies  for  each  of  the  six  Global  Vigilance,  Reach  and  Power 
(i.e.,  Globa!  Engagement)  core  competencies  that  the  Air  Force  has  identified  as  key  to  military 
dominance  in  the  post  cold  war  threat  environment.  In  particular,  ensuring  that  critical 
information  gets  to  warfighters  quickly  and  on  demand  is  fundamental  to  attaining  every  one  of 
the  six  competencies  and  to  success  in  the  post  cold  war  threat  environment. 

To  achieve  this  fundamental  information  capability,  the  military  is  increasingly  looking  to 
space.  Space  systems  are  becoming  vital  conduits  and  collectors  of  information.  Space 
systems  are  providing  global  communications,  precision  navigation,  weather,  warnings,  and 
intelligence,  surveillance,  and  reconnaissance  (ISR)  data  to  military  leaders,  planners, 
operators,  and  weapons  systems  in  near  real-time  and,  more  recently,  in  real  time.  The 
criticality  of  this  data  and  the  importance  of  receiving  it  quickly  cannot  be  over-emphasized. 
Accordingly,  the  advancement  and  transformation  of  space  systems  and  operations  has  become 
a  top  priority,  viewed  as  integral  to  achieving  the  Global  Engagement  core  competencies. 

Development  of  the  Space-Based  Infrared  System  (SBIRS)  of  satellites  is  an  example  of  the 
transition  the  space  community  is  undergoing  as  the  Air  Force  looks  to  the  space  community 
and  space  technologies  to  help  it  achieve  its  Global  Engagement  vision.  Using  SBIRS 
technology,  space  personnel  will  contribute  to  military  operations  in  ways  never  before  possible. 
However,  its  introduction,  along  with  the  myriad  of  other  changes  facing  the  space  community, 
mean  that  space  personnel  must  adapt  in  a  number  of  ways  —  that  they  must  learn  to  process 
increasing  amounts  of  information,  perform  new  tasks,  and  increasingly  support  the  operational 
community  and  understand  the  larger  and  more  complex  picture  within  which  they  are  operating. 
It  is  critical  that  training  keep  pace  with  these  changes  and  prepare  personnel  for  the  specific 
types  of  demands  they  will  face. 

The  criticality  of  skilled  task  performance,  and  therefore  of  effective  training,  is  evidenced  by 
the  potential  consequences  of  error.  In  the  realm  of  satellite  operations,  a  single  mistake  or  slow 
response  can  be  extremely  costly,  resulting  in  a  significant  reduction  in  a  satellite’s  lifespan  or  in 
the  loss  of  a  satellite,  which  can  cost  more  than  $1 .5  billion.  Even  the  temporary  loss  of  a 
satellite  is  extremely  serious  and  will  only  become  more  so  as  our  dependence  on  space 
technology  grows.  For  example,  such  a  loss  can  shut  down  large  sections  of  the  national  and 
world  economy.  Insight  into  this  potential  for  disruption  was  gained  in  May  of  1998  when  service 
to  more  than  80%  of  the  United  States’  approximately  45  million  pagers  was  interrupted  for  more 
than  24  hrs  due  to  a  malfunction  of  the  Hughes/  PanAmSat  Galaxy  IV satellite  (Gittien,  1998).  In 
March  1999,  the  GE-3  telecommunications  satellite,  which  serviced  most  of  North  America, 
suffered  an  anomaly  causing  widespread  disruption  of  television  and  radio  services,  including 
the  CNN  network,  Public  Broadcasting  Service,  and  Associated  Press  (CNN,  12  March  1999). 
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Satellite  anomalies  and  outages  can  similarly  impact  military  operations  in  significant  ways. 
The  reliance  of  military  operations  on  near  real-time  and,  in  some  cases,  real-time  information  is 
growing,  especially  as  the  military  moves  toward  its  goal  of  sensor-to-shooter  information 
delivery.  This  increasing  dependence  can  be  seen  in  the  considerable  role  played  by  space 
systems  during  the  relatively  recent  war  in  the  Balkans.  Space  systems  provided  battle  damage 
assessment,  making  possible  a  high  ops  tempo  and  a  rapid  retargeting/replanning  cycle;  they 
provided  weather  forecasting  for  planning;  the  Multi-Source  Tactical  System  in  bomber  cockpits 
displayed  space-derived  intelligence  that  allowed  bombers  to  be  retargeted  en  route  and  to  be 
used  in  the  close  air  support  (CAS)  of  ground  activity;  they  provided  the  global  positioning 
system  (GPS)  needed  for  all-weather  attacks  around  the  clock  and  for  synchronizing  datalinks; 
and  they  provided  an  extensive  global  communications  network  (Amrine,  2000). 

The  work  of  space  personnel  is  thus  becoming  increasingly  critical  to  military  operations. 

The  costs  of  the  space  systems  they  manage  are  growing,  and  the  cost  of  losing 
communications  with  any  given  satellite  for  even  a  short  period  of  time  is  growing.  Accordingly, 
while  efforts  are  made  to  improve  the  robustness  of  advanced  satellite  technology,  parallel 
efforts  must  be  made  to  improve  the  training  of  space  personnel.  Ensuring  the  proficiency  of 
space  personnel  during  this  period  of  rapid  transformation  is  essential. 

Training  in  Space  Operations:  Keeping  Pace  with  Change 

A  2002  article  in  Air  &  Space  Power  Chronicles  notes  that  although  Air  Force  Space 
Command  (AFSPC)  has  a  well-structured  and  generally  effective  training  program,  an  evaluation 
of  incident  data  from  the  50th  Space  Wing  (50  SW)  at  Schriever  AFB  suggests  a  need  for 
improvements  (Gottschalk  &  Gilchrist,  2002).  Incidents  evaluated  were  those  that  led  to  a 
satellite  outage  or  nonoperational  period.  It  was  found  that  38  of  a  total  of  60  mishaps  occurring 
during  a  five-year  period  (1997-2001)  were  caused  by  “operator  error  requiring  retraining”.  More 
specific  causes  included  inattention  to  detail,  incorrect  procedures,  and  lack  of  checklist 
discipline. 

These  incidents,  combined  with  the  potential  gravity  of  their  repercussions,  clearly  indicate 
that  there  is  room  for  improving  training  in  space  operations.  Furthermore,  significant  attention 
should  be  paid  to  training  whenever  operations  undergo  major  change.  At  a  basic  level,  such 
change  necessitates  the  development  of  training  curriculum  and  materials,  competency 
measures,  training  facilities,  and  much  more.  However,  the  increasing  complexity  of  space 
systems  and  of  the  operations  environment  in  which  space  personnel  work  calls  for  more  than 
updating  basic  training  resources.  It  demands  the  provision  of  training  resources  that  are  suited 
to  preparing  space  operations  personnel  for  the  complex,  dynamic,  information  rich,  and 
workload  intensive  environment  in  which  warfighters  are  increasingly  required  to  work. 

SBIRS  is  a  prime  example  of  new  space  technology  that  demands  these  types  of  enhanced 
training  resources.  SBIRS  is  being  developed  and  implemented  across  a  number  of  years  and 
likewise,  SBIRS  training  resources  are  being  developed  gradually  and  upgraded  in  parallel. 
These  training  resources  will  need  to  take  into  account  the  changing  work  environment  of 
satellite  personnel.  To  do  so,  they  may  need  to  undergo  dramatic  changes  to  meet  new  training 
objectives. 

Possibly  as  result  of  this  phased  development  of  SBIRS  training  facilities,  SBIRS  personnel 
are  faced  with  what  many  consider  a  training  resource  deficiency.  Specifically,  SBIRS 
simulation  training  facilities  exclude  certain  system  applications  that  are  used  by  operators  in  the 
satellite  operations  center  (SOC).  As  a  primary  example,  SBIRS  simulation-based  training 
facilities  lack  certain  important  applications  that  operators  use  in  the  SOC.  Consequently, 
procedures  often  must  be  skipped  or  pantomimed  by  trainees.  These  excluded  applications  are 
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basically  support  applications  that  operators  use  in  addition  to  the  core  set  of  crew  workstations, 
and  include  the  following: 

•  Space  weather  information  application 

•  The  New  Tactical  Forecast  System  (N-TFS)  for  terrestrial  weather 

•  Electronic  Scheduling  Dissemination  (ESD) 

•  The  Defense  Message  System  (DMS) 


APPROACH 

To  address  the  afore-mentioned  shortcomings  of  SBIRS  simulation  training  facilities,  a 
training  system  that  will  augment  and  enhance  training  provided  using  those  facilities  was 
designed.  The  objectives  of  this  design  are  twofold:  to  allow  operators  to  obtain  training  on  the 
use  of  space  operations  support  applications  that  are  currently  absent  from  SBIRS  training 
facilities,  and  to  provide  instructors  and  trainees  with  training  support  tools  that  will  enhance  the 
acquisition  of  trainee  expertise. 

Training  Requirements  Analysis 

To  develop  the  training  system  design,  a  training  requirements  analysis  was  conducted.  It 
was  clear  that  training  requirements  associated  with  the  absent  support  applications  listed  above 
were  not  being  adequately  met.  However,  it  was  important  to  determine  exactly  what  types  of 
skills  and  knowledge  associated  with  each  system  would  need  to  be  targeted  by  a  training 
system,  and  to  what  proficiency  level.  This  information  was  provided  largely  by  the  SBIRS  job 
performance  requirements  lists  (JPRLs).  These  lists  specified  that  SBIRS  operators  should  be 
proficient  in  a  number  of  skill  and  knowledge  areas  that  were  associated  specifically  with  the  use 
of  an  absent  support  applications.  For  example,  the  following  skill  and  knowledge  areas  are 
specified  in  the  SBIRS  JPRLs: 

•  Respond  to  Severe  Weather/Natural  Disaster  Notifications 

•  Identify  AFSCN  Equipment 

•  Perform  AFSCN  Configuration  Actions 

•  Respond  to  AFSCN  Malfunction 

•  Perform  Non-Routine  AFSCN  Scheduling 

•  Perform  Operational  Reporting  (reports  of  relevance  to  SBIRS  systems  personnel  include 
SOR,  OPSCAP,  and  MIR  reports) 

•  Perform  Electromagnetic  Interference  (EMI)  Procedures  (to  include  submission  of  EMI  report) 

•  Identify  Orbital  Mechanics/Space  Weather 

•  Respond  to  Severe  Weather/Natural  Disaster  Notifications  within  10  min 

•  Submit  SOR  within  2  hrs  of  initiation  of  hardcopy  reporting  and  within  1 5  min  for  an  L&PI 
generation  when  no  voice  communications  with  missile  C2  agency  exist 

•  Perform  Non-Routine  AFSCN  Scheduling  within  5  min  of  receipt  of  anomalous  information 

Thus,  a  number  of  SBIRS  job  performance  job  performance  requirements  (JPRs)  are 
associated  with  use  of  the  absent  support  applications.  In  addition  to  reducing  the  quality  of 
training,  this  mismatch  between  job  requirements  and  training  resources  can  potentially  create  a 
serious  staffing  shortage  if  it  delays  the  mission  ready  (MR)  certification  of  needed  personnel. 
Furthermore,  personnel  do  not  become  as  proficient  at  performing  tasks  in  the  absence  of 
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associated  training  resources  as  they  would  otherwise.  Without  the  absent  support  applications, 
SBIRS  training  facilities  do  not  support  acquisition  of  certain  basic  job  requirements  and,  by 
implication,  do  not  prepare  trainees  to  perform  their  jobs  as  proficiently  as  they  could. 

The  training  requirements  analysis  additionally  included  interviews  with  subject  matter 
experts  (SMEs).  SMEs  emphasized  that  training  facilities  need  to  support  the  requirement  that 
simulation-based  training  facilities  mirror  the  operations  center.  This  parallelism  makes  it 
possible  for  personnel  to  practice  complete  procedures  from  start  to  finish  and  develop  task 
performance  routines  that  span  across  multiple  systems.  As  a  result,  they  are  able  to  perform 
those  procedures  and  routines  during  real  world  operations  without  utilizing  significant  cognitive 
resources.  In  other  words,  complex  procedures  become  automatized  or  proceduralized,  and 
more  cognitive  resources  are  available  for  dealing  with  other  aspects  of  the  task  environment. 
This  allows  the  operator  to  devote  cognitive  resources  to  the  many  other  tasks  they  have  to 
perform. 

As  part  of  the  training  requirements  analysis,  survey  data  were  obtained  from  satellite 
operations  SMEs.  These  data  suggested  that  SBIRS  simulation-based  training  would  benefit 
from  support  tools  that  help  instructors; 

•  detect  trainee  errors, 

•  provide  trainees  with  one-on-one  attention  and  guidance,  and 

•  tailor  or  select  scenarios  to  better  meet  the  training  needs  of  various  groups  of  trainees 

In  addition  to  basing  the  training  system  design  on  the  results  of  the  training  requirements 
analysis,  the  training  environment  and  the  ways  in  which  the  training  system  could  be  used 
within  it  were  considered.  There  are  three  major  challenges  associated  with  the  training 
environment  that  the  training  system  design  must  address.  The  first  of  these  is  the  technology 
currently  used  to  run  and  store  exercise  scenarios.  The  training  system  will  need  to  be  able  to 
run  in  a  synchronized  manner  with  existing  training  suite  workstations  during  simulation  training 
exercises.  The  second  challenge  is  related  to  the  first  and  involves  designing  TESS  so  that  it  is 
consistent  with  the  ways  in  which  instructors  conduct  training.  For  example,  it  has  to  support 
instructors  in  the  development  and  execution  of  scenarios  that  are  consistent  with  the  scenarios 
currently  used  in  SBIRS  simulation  training  facilities.  The  third  challenge  has  to  do  with 
limitations  on  the  ways  in  which  training  can  be  presented  during  real-time  simulation  exercises. 
For  example,  real  time  expert  advice,  context-specific  explanation,  and  feedback  associated  with 
error  detection  were  all  identified  as  useful  training  strategies,  but  each  must  be  implemented  it 
in  a  way  that  does  not  interfere  with  time-pressured  task  performance. 

To  meet  these  challenges  and  address  all  identified  design  considerations  adequately,  the 
initial  training  system  design,  described  below,  will  be  subjected  to  additional  design  iterations 
based  on  feedback  from  SBIRS  training  personnel  and  technical  staff.  Feedback  will  be 
obtained  on  multiple  iterations  of  this  design  and  subsequently  on  iterative  versions  of  the 
functional  training  system  once  the  implementation  stage  of  development  is  reached.  Our  goals 
in  these  iterative  feedback  sessions  are  primarily  to: 

•  educate  SMEs  about  the  approach  and  technology  we  are  utilizing  so  they  can  better 
evaluate  the  system  and  make  informed  suggestions  and 

•  obtain  SME  feedback  about  iterative  versions  of  the  training  system  design. 

Other  means  of  meeting  these  challenges  and  addressing  considerations  adequately 
involve  the  appropriate  and  informed  use  of  technology.  For  example,  speech  synthesis 
technology  has  evolved  sufficiently  to  be  considered  as  a  means  of  providing  situation-specific 
explanation  aurally  without  distracting  trainees  from  tasks  that  involve  processing  primarily  visual 
information.  In  addition,  agent  technology  will  be  used  to  support  the  real-time  detection  of 
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errors  and  evaluation  of  context  in  order  to  provide  a  variety  of  context-specific  support.  This 
technology  is  described  in  the  section  below. 

TESS  Cognitive  Agents 

The  training  system  design  features  cognitive  agents  developed  using  CHI  Systems’ 
cognitive  agent  development  toolkit,  iGEN  (Zachary,  Ryder,  &  Hicinbothom,  1998).  The  role  of 
iGEN  cognitive  agents  within  the  TESS  conceptual  design  is  to  provide  performance  tracking, 
real  time  feedback,  and  situated  support  capabilities.  iGEN  provides  both  a  cognitive 
architecture  for  building  the  agents,  known  as  COGNET,  and  an  execution  engine,  known  as 
BATON,  that  allows  the  agents  to  interact  with  users  and  the  external  environment  in  real  time. 

In  effect,  iGEN  is  used  to  transform  a  cognitive  representation  of  task  performance  into  an 
executable  agent.  Thus,  it  is  especially  appropriate  for  developing  agents  that  must  track  or 
emulate  cognitive  task  performance.  In  addition,  COGNET  is  well  suited  to  supporting  the 
elicitation  and  representation  of  SME  domain  knowledge  and  expertise,  and  will  be  used  for  this 
purpose  during  the  development  of  the  TESS  cognitive  agents. 

The  cognitive  architecture  and  modeling  framework  embodied  by  the  iGEN  approach  to 
agent  development  is  influenced  principally  by  the  work  of  cognitive  scientist  Alan  Newell  (e.g., 
Card,  Moran,  and  Newell,  1983;  Newell,  1990).  In  its  simplest  form,  Newell’s  Unified  Theory  of 
Cognition  breaks  human  information  processing  into  three  parallel  macro-level  mechanisms  -- 
perception,  cognition,  and  motor  activity  -  and  attempts  to  characterize  them  in  terms  of  general 
principles  derived  from  years  of  cognitive  science  research.  Consistent  with  Newell’s  theory, 
iGEN  is  used  to  represent  perception,  decision-making,  and  action  and  link  them  within  a 
common  architecture  defined  by  cognitive  principles  about  how  information  is  stored,  retrieved, 
and  used,  and  how  these  processes  are  affected  by  human  resource  limitations. 

iGEN  consists  of  three  primary  components: 

Theoretical  framework  -  COGNET;  a  model  of  human  information  processing  architecture, 
derived  from  multiple  component  theories,  and  a  set  of  theoretically-derived  principles  of 
operation 

Description  language  -  CEL,  the  COGNET  execution  language;  a  formal  way  of  representing 
domain-specific  knowledge  in  terms  of  the  COGNET  information  processing  architecture  and 
principles  of  operation 

Behavioral/Cog nitive  simulation  tools  —  BATON;  an  executable  cognitive  architecture  associated 
with  a  larger  set  of  graphical  tools  that  support  the  authoring,  editing,  testing,  and  debugging  of 
cognitive  agents  using  CEL. 

The  iGEN  perceptual  mechanism  receives  information  from  the  outside  world,  translates  it 
into  symbolic  information  and,  via  perceptual  demons,  and  stores  it  in  a  local  information  store 
(i.e,,  a  blackboard  representation  of  memory)  that  is  accessed  by  both  the  perceptual  and 
cognitive  mechanisms.  iGEN  cognitive  processes  manipulate  the  information  on  the  blackboard, 
using  existing  procedural  knowledge  and  metacognitive  controls  embodied  in  the  task  models 
(i.e.,  tasks  are  represented  as  sets  of  procedures  organized  into  goal  hierarchies).  The  tasks 
can  modify  the  blackboard  as  part  of  their  reasoning  processes,  and  can  also  invoke  actions  that 
are  passed  back  to  the  external  environment/outside  world. 

The  COGNET  architecture  within  iGEN  includes  mechanisms  that  support  metacognitive 
activities  such  as  prioritizing  and  attention/task  switching.  These  mechanisms,  or  metacognitive 
controls,  include  context-sensitive  triggering  conditions  and  priority  formulas  that  influence  the 
order  in  which  cognitive  tasks  are  executed  and  the  ways  in  which  concurrently  'triggered’  tasks 
are  executed.  In  addition,  the  COGNET  architecture  specifies  a  metacognitive  blackboard 
where  knowledge  about  the  status  of  ongoing  activities  is  maintained.  iGEN  thus  supports  the 
representation  of  situated  goal  and  attention  shifting  in  complex,  multi-task  performance 
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situations.  Consequently,  iGEN  agents  behave  in  a  much  more  flexible,  non-deterministic  way 
than  task  network  models,  in  which  the  flow  of  tasks  must  be  predetermined. 

iGEN  has  been  commercially  available  for  more  than  four  years,  was  used  in-house  for  a 
number  of  years  prior,  and  has  been  described  as  comparable  with  other  leading  cognitive 
architectures  developed  for  building  agents  for  use  in  complex  domains  (e.g.,  Pew  &  Mavor, 
1998).  This  cognitive  agent  technology  has  been  used  to  develop  instructional  agents  that 
support  simulation-based  training  in  the  domains  of  anti-air  warfare,  ground  tactical  air  control, 
ship  control/piloting,  and  uninhabited  aerial  vehicle  control  (e.g.,  EBS,  2000;  Ryder  et  al.,  2000; 
Ryder,  Scolaro  &  Stokes,  2001;  Zachary,  et  al.,  1998),  and  to  develop  synthetic  teammates  that 
support  the  simulation-based  training  of  naval  air  wing  teams  (e.g.,  Zachary  et  al.,  2001).  In  the 
course  of  developing  this  training  system,  we  propose  to  reuse  major  components  of  our  existing 
iGEN  and  BATON  software.  This  strategy  will  allow  for  a  reduced  software  development  cost, 
direct  leveraging  off  many  years  of  research  and  development  into  building  executable  cognitive 
agents,  and  a  reduced  overall  risk  in  the  software  development  effort. 

RESULTS:  PHASE  I  CONCEPTUAL  DESIGN 

In  this  section,  we  describe  the  initial  design  of  the  training  system  we  are  developing  to 
augment  and  enhance  simulation-based  training  for  SBIRS  personnel.  This  system,  called  the 
Training  Environment  Support  System,  or  TESS,  has  the  primary  objectives  of  providing 
personnel  with: 

•  the  opportunity  to  perform  tasks  during  training  in  the  same  way  they  would  perform  them 
during  real  world  operations; 

•  opportunities  to  learn  how  to  interact  with  support  systems  and  use  their  information  in  a 
training  environment  rather  than  on  the  job;  and 

•  experience  responding  to  alerts  and  warnings  generated  by  those  support  systems. 

TESS  is  intended  to  augment  and  enhance  SBIRS  simulation-based  training  by  adding 
clones  of  SOC  support  applications  to  SBIRS  training  facilities,  and  by  providing  trainees  and 
instructors  with  training  support  tools  and  resources.  The  training  application  will  emulate 
support  applications  that  are  currently  absent  from  SBIRS  training  facilities  -  the  DMS,  N-TFS, 
ESD,  and  space  weather  support  applications  -  and  will  feature  training  support  tools.  The 
instructor  application  will  feature  instructor  support  tools  including  training  scenario 
development,  monitoring,  and  management  tools. 

The  TESS  training  application  will  be  installed  on  a  single  PC  within  each  training  facility, 
and  this  PC  will  be  networked  with  the  system  on  which  the  TESS  instructor  application  resides. 
This  latter  system  will  either  be  the  same  system  currently  used  to  manage  training  or  it  will  be 
an  adjacent  networked  system.  Alternatively,  it  may  be  necessary  to  recreate  all  scenarios  for 
use  by  the  TESS  application  alone,  and  TESS  would  supersede  the  existing  training 
management  software. 

TESS  additionally  will  be  integrated  with  training  suite  workstations.  This  will  allow  TESS  to 
detect  and  react  to  changes  in  the  simulation  environment,  such  as  the  occurrence  of  a  satellite 
anomaly,  and  it  will  allow  TESS  to  adapt  to  track  and  respond  to  the  progression  of  the  exercise 
scenarios.  As  described  in  more  detail  below,  a  performance  support  toolkit  called  ADEPT 
(Agent-Based  Decision  Enabling  and  Performance  Tracking)  that  is  being  developed  by  CHI 
Systems  for  use  on  SBIRS  SOC  workstations  will  be  integrated  with  TESS  to  facilitate  this 
linkage  with  the  training  suite  workstations. 

The  TESS  Training  Application 
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The  TESS  training  application  is  envisioned  as  a  multi-use  application.  Thus,  the 
application  can  be  used  in  basic  application  mode  to  provide  trainees  with  high  fidelity  versions 
of  the  SOC  applications  and  thereby  a  more  complete  operational  context  (see  Figure  1). 
Alternatively,  a  user  can  toggle  to  the  training  support  mode.  In  the  training  support  mode,  the 
cloned  applications  are  reduced  to  one-third  of  the  display  in  order  to  provide  trainees  with  a 
comprehensive  training  support  environment  (see  Figure  2). 
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Figure  1.  The  TESS  training  application  in  the  basic  application  mode. 


This  latter  training  support  mode  is  intended  to  expedite  the  development  of  proficiency  with 
the  SOC  support  applications  contained  within  TESS.  It  is  also  designed  to  help  trainees 
acquire  a  bigger  picture’  understanding  of  when  and  how  to  use  the  cloned  support  applications, 
including  an  awareness  of  the  sometimes  subtle  relationships  among  the  applications  (e.g.,  a 
space  weather  alert  may  necessitate  a  schedule  change  request  due  to  the  loss  of  an  asset 
caused  by  a  geomagnetic  anomaly). 
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Figure  2,  The  TESS  training  application  in  the  training  support  mode, 

TESS  trainee  support  tools  and  resources  available  to  trainees  when  TESS  is  used  in  the 
training  support  mode  will  include  the  following; 

•  TESS  cognitive  agent  -  the  TESS  cognitive  agent  will  track  and  assess  trainee  performance. 

Iff,—  a?ent  ouJPut  ~  trainees  will  be  given  real  time  feedback  on  their  performance 
as  well  as  context-sensitive  explanations  and  guidance. 

•  Managed  Exploratory  Learning  (MEL)  -  TESS  will,  via  the  TESS  cognitive  agent,  provide 
expert  mentoring  and  guidance  that  is  tailored  to  the  current  situation  within  an  ongoing 
training  exercise.  Guidance  will  change  based  on  whichever  clone  application  is  active. 

’  ^Idt®  De?S/0/1  Enabling  and  Performance  Tracking  (ADEPT)  decision  support  tools  - 
ADEPT  is  a  decision  support  toolset  CHI  Systems,  Inc.  is  developing  for  satellite  operators  to 
f  Se it  th®  S?c  as  Part,of  a  Phase  li  SBIR  effort.  Reusing  the  ADEPT  application  in  training 
facilities  by  integrating  it  with  TESS  will  give  trainees  access  to  the  ADEPT  tools,  including 

o  AWE  tool-  trainees  can  access  this  tool  to  view  a  prioritized  list  of  alerts,  warnings 

and  eYe"!l.(AVVEs).-  Am°ng  other  things,  AWE  codes  are  defined,  time  limits  on  time 
critical  AWEs  are  given,  and  course  of  action  recommendations  are  provided. 
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Messaging  tool-  this  tool  will  allow  instructors  to  give  trainees  individualized  attention 
and  guidance. 

o  Satellite  position  viewing  tool  -  this  visualization  tool  provides  trainees  with  a  3-D  view 
of  satellite  locations  relative  to  the  sun,  earth,  moon,  penumbrae,  and  umbrae,  and 
facilitates  understanding  orbital  mechanics. 

•  Self-directed  training  tool  -  when  TESS  is  used  in  standalone  mode,  trainees  can  choose 
from  a  categorized  list  of  training  events  that  are  specific  to  the  TESS  clone  applications  and 
train  using  TESS  in  the  absence  of  an  instructor  or  when  personnel  are  not  available  to 
support  a  full  training  exercise. 

The  TESS  Instructor  Application 

The  TESS  instructor  application  will  be  used  to  give  instructors  tools  and  resources  to  help 
them  manage  training  sessions.  The  TESS  instructor  application  (see  Figure  3)  features  five 
primary  modes  of  use  -  automated,  manual,  scenario  editor,  analysis,  and  help.  It  is  designed 
to  provide  varying  levels  of  automated  control  over  the  execution  of  a  training  session,  and 
additionally  offers  manual  control  for  situations  in  which  automated  control  is  not  desired. 


Figure  3.  The  TESS  instructor  application. 
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The  proposed  set  of  TESS  instructor  support  tools  provided  within  this  application  are  listed 
below, 

•  Scenario  management  tools,  including: 

o  Automated  scenario  controls  -  instructors  will  be  able  to  select  and  launch  predefined 
scenarios  from  a  menu.  Each  scenario  will  be  associated  with  a  script  that  may  be 
edited  and  printed. 

o  Scenario  development  controls  -  instructors  will  use  these  controls  to  organize  multiple 
training  events  into  a  single  training  scenario  and  script.  These  controls  will  include  a 
scenario  development  filter  that  will  allow  instructors  to  view  and  select  from  training 
events  that  are  associated  with  a  particular  training  objective.  Example  objectives 
include  various  JPRs,  skills  such  as  proactivity,  crew  coordination,  and  ‘comm’ 
discipline,  and  proficiency  using  various  applications. 

o  Manual  event  insertion  controls  -  instructors  will  be  able  to  insert  events  into  the 
training  environment  one  at  a  time  instead  of  using  scenario  controls  to  present  a 
predefined  series  of  events. 

o  Event  editing  controls  -  these  controls  allow  instructors  to  tailor  selected  events  with 
respect  to,  for  example,  severity,  satellite  system  affected,  ground  system  affected, 
start  time,  and  duration. 

•  Training  session  viewer- this  tool  allows  instructors  to  view  time-stamped  interactions 
between  the  trainee  and  the  TESS  clone  applications  during  scenario  execution,  as  observed 
by  a  TESS  cognitive  agent.  The  tool  additionally  features  the  detection  and  report  of  trainee 
errors  by  a  cognitive  agent. 

•  Messaging  tool  -  this  tool,  made  available  via  the  ADEPT  toolkit  mentioned  above,  will  allow 
instructors  to  give  trainees  individualized  attention  and  guidance. 

•  Performance  evaluation  capabilities  -  instructors  will  have  the  ability  to  specify  performance 
measures  to  be  collected  for  a  given  event  using  the  event  editing  controls.  These  measures 
will  be  collected  using  the  TESS  cognitive  agents  and  the  results  will  be  entered  into  a 
database  and  subsequently  accessed  by  the  TESS  instructor  application  that  instructors  will 
use  to  develop  and  print  training  summary  reports  for  training  exercise  participants. 

FUTURE  WORK 

Space  operations  are  rapidly  transforming,  and  these  changes  pose  a  constant  challenge  to 
training  effectiveness.  It  is  especially  challenging  to  adapt  training  to  fundamental  changes  in 
the  complexity,  stress,  dynamicism,  and  degrees  of  freedom  in  the  operations  environment. 
Adapting  to  such  changes  in  a  domain  in  which  operator  proficiency  is  vital  calls  for  the 
development  of  training  and  training  systems  that  take  advantage  of  advanced  technologies  and 
training  strategies  that  were  developed  to  optimally  support  performance  in  complex  and 
dynamic  operational  environments. 

The  design  effort  described  in  this  report  involves  the  utilization  of  training  strategies  and 
technologies  that  are  particularly  applicable  to  simulation-based  training  to  not  only  meet  basic 
training  needs,  but  to  also  facilitate  the  acquisition  of  expertise.  For  example,  training  strategies 
include  scaffolding  (e.g.,  via  agent-based  feedback  and  MEL)  and  real-time  mentoring  (via 
MEL).  Technology  being  utilized  to  these  ends  includes  agent-based  technology,  which  will  be 
used  to  track  trainee  performance  in  the  fest-paced  simulation  environment.  In  addition,  the  use 
of  synthesized  speech  to  convey  real-time  advice  and  feedback  will  be  evaluated  based  on 
feedback  received  on  the  TESS  demonstration. 

Future  work  will  involve  working  with  space  operations  training  personnel  to  iteratively  refine 
the  functionality  and  user  interface  design  of  TESS,  and  then  to  iteratively  develop  the  training 
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